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REMARKS 

The present application was filed on June 18, 2 001 with claims 1 through 
19, of which claims 1, 7, and 15 are independent claims. With the present response, 
Applicants propose to amend claims 1, 7, and 15. 
5 In the outstanding Office Action, the Examiner rejected all claims under 

35 USC § 103(a) as being unpatentable over Jain et al. (U.S. Patent No. 5,983,237, 
hereinafter, "Jain") in view of Brown (U.S. Patent No. 6,61 1,842, hereinafter, "Brown"). 

Amendment to the Disclosure 
10 Applicants have amended the disclosure to add a cross-reference to a 

related application. 

Amendments to Claims 

Applicants have made clarifying amendments to independent claims 1 , 7, 
15 and 15. These claim changes are supported, inter alia, by FIGS. 2, 4, and 17A through 
17C and associated text of Applicants' specification. 

Rejection to Independent Claims 1, 7, and 15 Under 35 USC $ 103(a) 
The Examiner rejected independent claims 1, 7, and 15 under 35 USC 
20 § 103(a) as being unpatentable over Jain in view of Brown. The Examiner asserted that 
Jain teaches all elements of the independent claims except assembling a user profile 
based on categorization. The Examiner asserted that Brown teaches assembling a user 
profile based on the categorization. See pages 2 and 3 of the outstanding Office Action. 

Each of amended and originally filed independent claims 1, 7, and 15 
25 contain the following limitations: (1) each multimedia item having two or more disparate 
modalities, the disparate modalities being at least one or more visual modalities and one 
or more textual modalities; (2) creating a visual feature vector for each of the visual 
modalities and a textual feature vector for each of the textual modalities; and (3) 
concatenating the visual feature vectors and the textual feature vectors into unified 
30 feature vectors. Applicants respectfully submit that Jain or Brown, alone or in 
combination, do no not disclose limitations (1), (2), and (3). 
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First, Applicants respectfully submit that Jain deals with retrieval of 
images (see, e.g., col. 7, lines 33-38 of Jain), and does not disclose that the image files 
have textual modalities, as in limitation (1) of independent claims 1, 7, and 15. 

Second, Applicants respectfully submit that Jain is a term-based image 
5 retrieval system that does not create textual feature vectors. Instead, Jain uses "term- 
arrays" to determine images to return in response to a user query. Furthermore, for the 
sake of argument, even if the "term-arrays" can be considered to be "textual feature 
vectors" in accordance with the present invention, Jain does not disclose concatenating 
textual and visual feature vectors into unified feature vectors. 
10 For instance, Applicants read Jain as providing a "victionary" that will 

retrieve images in response to a term-based or image-based query. See Abstract of Jain 
and FIG. 8 of Jain. Jain does disclose feature vectors. Specifically, Jain states the 
following: 

A feature vector (FV) describes a visual object. Vectors 
15 form a uniform base type for features representing image content. In a 

presently preferred embodiment, the primary data type in the VIR Engine 
is a (indexable) collection of feature vectors. A visual feature is any 
property of an image that can be computed using computer-vision or 
image-processing techniques. Examples of features are hue, saturation, 
20 and intensity histograms; texture measures such as edge density, 

randomness, periodicity, and orientation; and shape measures such as 
algebraic moments, turning angle histograms, and elongatedness. Some of 
these features are computed globally, i.e., over an entire image, and some 
are local, i.e., computed over a small region in the image. 

25 Thus, a feature vector comprises visual features computed using computer-vision or 
image processing techniques. Also, terms are associated with feature vectors and are 
used to search feature vectors. Jain states: "[t]he second aspect is the architecture of the 
visual dictionary, which associates a text term (a concept) with a set of visual feature 
vectors for each 'visual sense 5 of the term. Visual sense refers to a specific visual 

30 appearance that the term may take in an image." See Jain at col. 8, line 66 to col. 9, line 

3. Jain also states the following at col. 10, lines 27-34: 

FIG. 6 shows how the feature space 270 of an exemplary 
term may appear. Typically, the feature space is hyperspace, i.e., an n- 
dimensional space, where n»l. For ease in discussion, however, a two- 
35 dimensional example is used herein. In this hypothetical two-dimensional 



8 



Docket No.: YOR920010536US1 



feature space, there are three shaded regions 270, 274 and 276, each 
representing a different visual sense of the term. 

In other words, in Jain, the feature space relates text terms to visual feature vectors. 
However, Jain never discloses that textual feature vectors are ever created from textual 
5 modalities of a multimedia item or used to create a "unified feature vector," as in 
limitations (2) and (3) of Applicants 5 independent claims. 

For instance, FIG. 9 of Jain shows a method of creating a query structure 
from a user query. If the user query is a text query (box 356 = TERM), then the query 
structure comprises a term-array (step 372). If the user query is an image query (box 356 

10 = IMAGE), then feature vectors are computed (step 358) and placed into the query 
structure (step 364). However, the feature vectors are converted to text query terms in 
additional methods. For instance, in step 406 of FIG. 10A of Jain, the image query 
structure is sent to the "victionary." The victionary executes an association manager in 
step 686 of FIG. 15 of Jain. The association manager 686 passes a concept to an 

15 association manager, which generates synonyms. See step 710 of FIG. 15 and col. 18, 
lines 1 8-20 of Jain. A concept is one or more phrases that the dictionary is capable of 
handling. See col. 16, line 50 to col. 17, line 33 of Jain for the "Concept Manager." 
Thus, an image query is converted to a number of synonyms. The synonyms are used to 
search for visual feature vectors associated with images. See step 712 of FIG. 15 of Jain 

20 and block 3 1 4 of FIG. 8 of Jain. 

Applicants respectfully submit that Jain does not disclose creating "textual 
feature vectors" for textual modalities of a multimedia item, in accordance with the 
present invention, as there is no teaching or implication in Jain that such textual feature 
vectors are being created. As described above, Applicants read Jain as disclosing that 

25 term-arrays are used to search for visual feature vectors associated with images. 

For the sake of argument, even if the "term-arrays" can be considered to 
be "textual feature vectors" in accordance with the present invention, Jain does not 
disclose concatenating textual and visual feature vectors into unified feature vectors. If 
there is a "textual feature vector" in Jain (which Applicants respectfully state there is 

30 not), the textual feature vector could be used to examine a feature space corresponding to 
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visual feature vectors. However, a concatenation of textual and visual feature vectors is 
never performed in Jain. 

Therefore, Applicants respectfully submit that Jain does not disclose or 
imply limitations (1), (2), and (3) of independent claims 1, 7, and 15. 

As for Brown, Applicants respectfully submit that Brown also does not 
disclose limitations (1), (2), and (3) of independent claims 1, 7, and 15. Applicants read 
Brown as providing a computer system the generates user profile data reflecting 
underlying characteristics of user preferences. See Abstract of Brown. However, Brown 
does not disclose or imply determining textual or visual feature vectors from textual or 
visual modalities of a multimedia item or of concatenating the textual and visual feature 
vectors into unified feature vectors, as in limitations (1), (2), and (3) of independent 
claims 1, 7, and 15. 

Consequently, Applicants respectfully submit that independent claims 1, 7 
and 15 are patentable over Jain or Brown, alone or in combination. 

Rejections to Dependent Claims 2-6, 8-14, and 16-19 Under 35 USC 

SI 03(a) 

In the outstanding Office Action, the Examiner rejected dependent claims 
2-6, 8-14, and 16-19 under 35 USC §103(a) as being unpatentable over Jain in view of 
Brown. As argued above, Applicants respectfully submit that independent claims 1, 7, 
and 19 are patentable over Jain or Brown, alone or in combination. Because independent 
claims 1, 7,and 15 are patentable, dependent claims 2-6, 8-14, and 16-19, which include 
all limitations of their respective independent claims from which they depend, are also 
patentable. Furthermore, Applicants also assert that dependent claims 2-6, 8-14, and 16- 
19 recite patentable subject matter in their own right. 

Conclusion 

All of the pending claims, i.e., claims 1-19, are in condition for allowance 
and such favorable action is earnestly solicited. 
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If any outstanding issues remain, or if the Examiner has any further 
suggestions for expediting allowance of this application, the Examiner is invited to 
contact the undersigned at the telephone number indicated below. 

The Examiner's attention to this matter is appreciated. 



Respectfully submitted, 



Date: May 10, 2004 <>^0tert J. Mauri 

^wVttorney for Applicants 
Reg. No. 41,180 
Ryan, Mason & Lewis, LLP 
1300 Post Road, Suite 205 
Fairfield, CT 06824 
(203) 255-6560 
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